J 



® 



EuropSlsches Patsntamt 
European Patent Office 
Office europden dea brevets 





(g) Publication number: 0 571 180 A2 



EUROPEAN PATENT APPLICATION 



g) Application number : 93303838.2 
S) Date of fQing : 18.05.93 



@ int. CL 5 : H04N 7/137 



(So) Priority: 22.05.92 JP 155719/92 

(S) Date of publication of application : 
24.11.93 Bulletin 93/47 

(84) Designated Contracting States : 
DE FR GB 

© Applicant : SONY CORPORATION 
6-7-35 KHashlnagawa Shlnagawa-ku 
Tokyo 141 (JP) 



72) Inventor : Kondo, TetSMjIro, do Patents 
Division 

Sony Corporation, 6-7-35 Kltashlnagawa 
Shlnagawa, Tokyo 141 (JP) 

74) Representative : Targett, Kenneth Stanley et al 
D. Young & Co., 21 New Fetter Lane 
London EC4A 1DA (GB) 



(3) Digital data conversion equipment 

(g) Digital data conversion equipment, compris- 
es; means for analyzing plural input data and 
performing clustering depending on a distribu- 
tion state of the plural input data, means for 
generating class data associated with output 
data for each class on the basis of the plural 
input data and known output data, memory 
means for storing the class data at an address 
corresponding to the class, read-out means for 
reading out class data at an address corre- 
sponding to class information subjected to 
clustering based on the plural input data, and 
output data generating means for generating 
output data based on output class data of the 
read-out means. 
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This invention relates to a digital data conversion equipment and a method for the same, which are appli- 
cable to an interpolation of a thinned picture element in data conversion, up-conversion for converting a tele- 
vision signal with standard resolution into a television signal with high resolution and so on. 

There are generally two kinds of systems for converting a digital video signal. One of them is a system for 
5 converting a signal whose resolution is high with respect to the space or time or a signal having a large amount 
of information into a signal of a low resolution. The other is a system for converting, on the contrary, a signal 
whose resolution is low with regard to the space or time or a signal having a small amount of information signal 
of a high resolution. 

In the former case, a signal having inherently a large information amount is converted into a signal of a 
10 small information amount. For example, by properly thinning out a picture element information amount or 
f ieldtframe information, a signal of a low space/time resolution can be easily formed. 

The above example relates to what is called a down converter to, for instance, convert a video signal of 
a high definition (HD) system into a video signal of a standard definition (SD) system. Various kinds of techni- 
ques have already been proposed. 
is The latter case, contrarily, relates to an up conversion to, for instance, convert a video signal of the SD 

system to a video signal of the HD system. An example in which an electronic zooming process is executed 
or an enlargement of an image is performed is considered. In those examples, hitherto, information which in- 
herently lacks is interpolated by using an interpolation filter and the interpolated information is used. 

As still another example, there is a sub-sampling method for periodically thinning out pixel data in order 
20 to compress a recording/transmission data amount in the case where a capacity of the recording/ transmitting 
system is limited. In this case, the images thinned out are interpolated on the reproducing/receiving side by 
using an interpolation filter. 

However, there is a problem that the resolution of an output picture from obtained by the interpolation with 
a filter is degraded. For example, even if a HD television signal formed by interpolating a SD video signal by 
25 a filter, a HD component (high frequency component) which is not present in an input SD signal is not repro- 
duced. As a result, the spatial resolution of an output picture is lowered. 

The present invention is concerned with providing digital data conversion equipment and a method for the 
same capable of reproducing a high resolution component 

According to an aspect of the present invention, there is provided a digital data conversion equipment, com- 
30 prising: means for analyzing plural input data and performing clustering depending on a distribution state of 
the plural input data; 

means for generating class data associated with output data for each class on the basis of the plural 
input data and known output data; 

memory means for storing the class data at an address corresponding to the class; 
35 read-out means for reading out class data at an address corresponding to class information subjected 

to clustering based on the plural input data; and 

output data generating means for generating output data based on output class data of the read-out 

means. 

According to another aspect of the present invention, there is provided a digital data conversion method, 
40 comprising: 

training step for analyzing plural input data, performing clustering depending on a distribution state of 
the plural input data, generating class data associated with known output data for every class on the basis of 
the plural input data and the output data, and storing the class data into a memory depending on the class; 
step for clustering the plural input data and reading out class data at an address of the memory corre- 
45 sponding to the class; and 

step for generating output data based on the class data. The invention will now be more particularly de- 
scribed, byway of illustrative and non-limiting example, with reference to the accompanying drawings, in which: 
Fig. 1 is a schematic block diagram of a transmission system to which the invention can be applied; 
Fig. 2 is a schematic diagram showing a position relation of picture elements; 
so Fig. 3 is a block diagram of one example of a structure for generating a mapping table; 

Fig. 4 is a block diagram of another example of a structure for generating a mapping table; 
Fig. 5 is a block diagram of another embodiment; 

Fig. 6 is a schematic diagram showing a position relation of picture elements of a SD picture and a HD 
picture; and 

55 Fig. 7 is a block diagram of one example of a structure for generating a mapping table. 

Hereunder, one embodiment of this invention will be explained. This one embodiment transmits thinned 
and compressed data and reproduces a thinned picture element on the reception side. Fig. 1 shows such a 
transmission system as a whole. In Fig. 1, reference numeral 1 is an input terminal for digital video data to be 
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transmitted. 

Input digital video data is supplied to a sampling circuit 2, and picture element data positioned alternately 
is thinned out in the horizontal direction. As shown in Fig. 2, picture elements indicated at X in the array of the 
5 original picture elements show the thinned picture elements a result, with this thinning-out process, the data 
amount necessary for transmission is reduced to half. 

The output data of the sampling circuit 2 is supplied an encoder for highly efficient coding. For the highly 
efficient coding, orthogonal conversion coding such as DCT (Discrete Cosine Transform), ADRC (Dynamic 
Range Adaptive-type Coding) and so on, which are well known, can be adopted. With this encoder 3, the data 
10 amount to be transmitted is reduced. 

The output data of the encoder 3 is fed to a transmission processing circuit 4. The transmission processing 
circuit 4 performs processing such as error correction coding, frame formation and channel coding. Transmis- 
sion data is generated and an output terminal 5 of the transmission processing circuit 4. The transmission data 
is through a transmission line 6. The transmission line 6 is limited to a communication line and includes proo- 
fs esses of magnetic recording and reproduction in its meaning. 

Reception data is fed through an input terminal 7 to a reception processing circuit 8. The reception proc- 
essing circuit 8 performs processing such as decoding of channel coding, frame decomposition, and error cor- 
rection. The output of the reception processing circuit 8 is given to a decoder 9 for highly efficient coding. The 
decoded output of the decoder 9 is supplied to a selecting circuit 10 and a simultaneous output circuit 11. 
20 The simultaneous output circuit 11, as shown in Fig. 2, produces transmission picture element data a, b, 

c, and d, which are present at upper and lower positions and left and right positions with respect to a thinned 
picture element x to be interpolated, to a clustering circuit 12 and an interpolation data generating circuit 14, 
simultaneously. Output data from the clustering circuit 14, i.e., class information is given to a memory 13 as 
an address signal. 

25 A mapping table for data conversion formed in a manner mentioned later is stored in the memory 13. In 

this example, a mapping table including plural parameters is stored in the memory 13. A parameter read out 
from an address corresponding to the output data of the clustering circuit 12 is supplied to the interpolation 
data generating circuit The interpolation data generating circuit 14 provides interpolation data x by the calcu- 
lation of. 

30 x = w1a + w2b + w3c + w4d 

using transmission picture element data a, b, c f and d from the simultaneous output circuit 11 and parameters 

w1, w2, w3, and w4 from the memory 14. 

The interpolated data x is supplied to the selecting circuit 10. The selecting circuit 10 selects the output 

the decoder 9 when a transmission picture element is present, while the selecting circuit 1 0 selects interpolated 
35 data from the interpolation data generating circuit 14 at a position of a thinned picture element Consequently, 

decoded video data corresponding to reception data is provided at an output terminal 15 of the selecting circuit 

10. 

A mapping table formed in advance by training is stored in the memory 13. Fig. 3 shows a structure for 
forming the mapping table. In Fig. 3, a digital video signal is supplied to 21 and to a simultaneous output circuit 

40 22. It is desirable that the digital video signal is a standard signal taking into a count the generation of a mapping 
table. For example, for the video signal, a signal composed of a still picture with various patterns can be adopt- 
ed. As shown in Fig. 2, the simultaneous output circuit 22 supplies a data memory 23 and a clustering circuit 
24 simultaneously with data a x, which is a target picture element, and picture element data a, b, c, and d, which 
are present in upper and lower positions and left and right portions with respect to the data x. It is to be noted 

45 that actual value exists without any thinning of the target picture element at the time of the training shown in 
Fig. 3. 

The clustering circuit 24 carries out the clustering picture element data to generate class information as 
the clustering circuit 12 of Fig. 1 does. For clustering, clustering by gradation, clustering by a pattern, etc., can 
be used. In the use of the gradation, the number of classes becomes extremely large If picture element data 
so has eight bits. As a result, it is desirable that the bit number of each picture element is reduced with highly 
efficiency coding such as ADRC. For pattern use, plural patterns composed of four picture elements (for ex- 
ample, evenness, increase of a value in the right and upper direction, decrease of a value in the right and lower 
direction, etc.) are prepared, and the output data of the simultaneous output circuit 22 is classified into any 
one of the plural patterns. 

55 The output of the clustering circuit 24 is given to one input terminal 25a of a switching circuit 25. The output 

of a counter 26 is supplied to the other input terminal 25b of the switching circuit 25. The counter 26 generates 
addresses, which sequentially change, by counting clock CK. The output of the switching circuit 25 is supplied 
to the data memory 23 and a memory 28 for parameters as their addresses. 

Sample values of picture element a, b, c, d, and x are written into the data memory 23 with respect to ad- 
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dresses which are class information. For example, (a 10 , a^ .... a„o) with respect to the picture element data 

a. (bio, bao b„o) as to the picture element data b, (c, 0 , .... c^) with respect to the picture element data 

c, and (d 10 , 6^ .... d^) as to the picture element data d are stored in a certain address ADO of the data memory 
23. As for other addresses from the clustering circuit 24, picture element data is stored in the memory 23 sim- 
ilarly. 

Next, the switching circuit 25 is switched from the input terminal 25a to 25b, and the content of the data 
memory 23 is sequentially read out by an address from the counter 26. The read-out of the data memory 23 
is supplied to an arithmetic circuit 27 of the least square method. With this minimum square method, parame- 
ters w1 to w4 are obtained with minimum error. 

When attention is paid to one address, the following simultaneous equations are established with respect 
to this address: 

xl = wlal + w2bl ♦ w3cl ♦ w4dl 
x2 = wla2 + w2b2 + w3c2 + w4d2 



xn = wlan + v?2bn + w3cn ♦ w4dn 

Now, since x1 to xn, a1 to an, b1 to bn, d to cn, and dl to dn are known in advance, the parameters wl to 
w4 are obtained so that the square of the error for x1 to xn (actual values) is minimized. This applies to other 
addresses. 

The parameters w1 to w4 obtained at the arithmetic circuit 27 are written into a memory 28. A mapping 
table which has been written into the memory 28 is stored in the memory 13 of Fig. 1 . Therefore, the value of 
x, which is a thinned picture element, is produced at the interpolation data generating circuit 14 using the para- 
meters produced from the memory 13. 

For the mapping table, not only the above-stated parameters but also the one from which output data val- 
ues themselves are provided may be employed. In this case, the interpolation data generating circuit 14 in Fig. 
1 can be omitted. Fig. 4 shows a structure for forming the mapping table. Similarly to the structure of Fig. 3, 
plural picture element data made simultaneously is supplied to the clustering circuit whose output is supplied 
to a data memory 30 and a frequency memory 31 as an address. 

The read-out output of the frequency memory 31 is give to an adder 32 and added by +1. The output of 
the adder 32 is written into the same address of the memory 31. For the memories 30 and 31, each content 
of their addresses is cleared at zero as the initial stage. 

Data read from the data memory 30 is supplied to a multiplier 33 and multiplied by a frequency which is 
read out of the frequency memory 31 . The output of the multiplier 33 is given to an adder 34 and added to the 
input data x there. The output of the adder 34 is supplied to a divider 35 as a dividend. To the divider 35, the 
output of the adder 32 is fed as a divisor. The output of the divider 35 (quotient) becomes input data of the 
data memory 30. 

In the above-mentioned structure of Fig. 4, data x1 is directly written into the memory 30 and the value 
of a corresponding address of the memory 31 is brought to 1 , since the read outputs of the memories 30 and 
31 are zero. If this address is accessed once again later, the output of the adder 32 is 2, and the output of the 
adder 34 is (x1 + x2). As a result, the output of the divider 35 is (x1 + x2)/3, which is written into the memory 
30. On the other hand, the frequency z is written into the frequency memory 31. Further, when the above- 
mentioned address is accessed, the data of the memory 30 is updated to (x1 + x2 + x3V3. The frequency is 
also updated to 3. 

By carrying out the above-mentioned operation within a determined period, a mapping table is stored into 
the memory 30 so that data, which is present at that time, is output when a class is designated by the output 
of the clustering circuit In other words, when plural picture element data of an input video signal is given, a 
mapping' table can be formed so that data is output to correspond to its clustered data on the average. 

Another embodiment of the invention shown in Fig. 5 is for uj>-conversion of a SD video signal to a HD 
video signal. In Fig. 5, a digital SD video signal is supplied to a terminal indicated at 41. Examples of the SD 
video signal are a reproduction signal, a broadcast signal, etc, of SDVTR. The SD video signal is given to a 
simultaneous output circuit 42 whose output data is supplied to clustering circuit 43. The output of t he clustering 
circuit 43 is sent as an address signal to memories 44a to 44d where mapping tables M1 to M4 are stored. 
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Fig. 6 partially shows a relationship between a SD picture and a HD picture. In Fig. 6 t picture element data 
interpolated by circles O belongs to the SD picture, while picture element data indicated by crosses X belongs 
to the HD picture. For example, four picture element data y1 to y4 of the HD picture is generated from twelve 
5 picture element data of the SD picture. The mapping table M1 of the memory 44a is for generating picture ele- 
ment data y1, while the mapping tables M2, M3, and M4 are for generating picture element data y2, y3, and 
y4, respectively. 

The read-out outputs of the memories 44a to 44d are given to a selector 45. The selector 45 is controlled 
by the output of a selection signal generating circuit 46. A sampling clock of the HD picture is supplied from 

10 an input terminal 47 to the selection signal generating circuit 46. The four picture element data y1 to y4 is se- 
lected sequentially by the selector 45 and is supplied to a scanning conversion circuit 48. The scanning con- 
version circuit 48 generates picture element data of the HD picture in the order of raster scanning at an output 
terminal 49. A monitor for HD is connected to the output terminal 49 through a D/A converter (not shown). The 
number of picture elements of an output picture is four times that of picture elements of an input SD video signal. 

15 Fig. 7 shows one example of a structure for generating the mapping tables M1 to M4 stored in the memories 

44a to 44d. In Fig. 7, a digital HD video signal is supplied to an input terminal indicated at 51. It is desirable 
that the HD video signal is a standard-like signal taking into account the generation of the mapping tables. 
Actually, by taking a standard picture with a HD video camera or by recording a taken picture signal onto 
HDVTR, a HD video signal can be provided. 

20 The HD video signal is supplied to a simultaneous output circuit 52. The simultaneous output circuit 52 

simultaneously produces picture element data a to 1 and y1 to y4 having a relationship in positions shown in 
Fig. 6. The picture element data a to 1 is supplied to a clustering circuit 53. The clustering circuit 53 performs 
the classification of gradation, patterns, etc., as in the above-mentioned one embodiment. The output of the 
clustering circuit 53 is commonly given to mapping table generating circuits 54a to 54d. 

25 The picture element data y1 to y4 is supplied to the mapping table generating circuits 54a to 54d have the 

same construction. The one similar to the structure for obtaining average value as shown in Fig. 4 can be adopt- 
ed for the mapping table generating circuits 54a to 54d. In the case of the mapping table generating circuit 
54a, y1a is supplied in place of the picture element data x in Fig. 4. For the mapping table generating circuit 
54a, the same structure of Fig. 4 can be employed. In addition, with the use of parameters, the same structure 

30 as Fig. 3 may be used for mapping generating circuits 54a to 54d. 

Mapping tables showing the correlation between the HD video signal and the SD video signal are stored 
in the mapping table generating circuits 54a to 54d. In other words, when plural data of the SD video signal is 
given, a mapping table, which outputs picture element data of the HD video signal on the average correspond- 
ing to the one provided by clustering these a plural data, can be formed. This mapping table is stored in the 

35 memories 44a to 44d with the structure of Fig. 5. 

Although the above-mentioned one embodiment is an example where the up-conversion of the SD video 
signal to the HD video signal is made, the invention can be applied similarly to the enlargement of a picture, 
besides this embodiment. 

With the above arrangement, data transmitted with a thinning-out system can be received, and a thinned 
40 picture element can be interpolated without the deterioration of resolution. When picture element data lacking 
at the time of picture element enlargement is interpolated, the invention is applicable in a similar manner. Also, 
the above arrangement not only permits a video signal with standard resolution to be converted to that with 
high resolution but also allows a picture with high resolution to be displayed on a monitor. 

Having described specific preferred embodiments of the present invention with reference to the accom- 
45 panying drawings, it is to be understood that the invention is not limited to those precise embodiments, and 
that various changes and modifications may be effected therein by one skilled in the art 



Claims 

50 

1. Digital data conversion equipment, comprising: 

means for analyzing plural input data and performing clustering depending on a distribution state 
of said plural input data; 

means for generating class data associated with output data for each class on the basis of said 
55 plural input data and known output data; 

memory means for storing said class data at an address corresponding to said class; 
read-out means for reading out class data at an address corresponding to class information sub- 
jected to clustering based on the plural input data; and 

output data generating means for generating output data based on output class data of said read- 
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out means. 

2. A digital data conversion method, comprising: 

5 training step for analyzing plural input data, performing clustering depending on a distribution state 

of said plural input data, generating class data associated with known output data for every class on the 
basis of said plural input data and said output data, and storing said class data into a memory depending 
on said class; 

step for clustering the plural input data and reading out class data at an address of the memory 
10 corresponding to the class; and 

step for generating output data based on said class data. 

3. Digital data conversion equipment as claimed in claim 1, wherein said class data is output data itself. 
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4. Digital data conversion equipment as claimed in claim 1, wherein said class data is parameter data and 
wherein said output data generating means has arithmetic means for said parameter data and said input 
data. 

5. A digital data conversion method as claimed in claim 2, wherein said class data is output data itself. 

20 6. A digital data conversion method as claimed in claim 2, wherein said class data is parameter data and 
wherein the step for generating said output data performs an arithmetic operation for said parameter data 
and said input data. , 

7. Digital data conversion equipment, comprising: 
25 means for analyzing plural input data of a first standard system and performing clustering depend- 

ing on a distribution state of said plural input data; 

means for generating class data associated with output data for each class on the basis of t he plural 
input data of said first standard system and known output data of a second standard system; 
memory means for storing said class data at an address corresponding to said class; 
30 read-out means for reading out class data at an address corresponding to class information sub- 

jected to clustering based on the plural input data of said first standard system; and 

output data generating means for generating output data based on output class data of said read- 
out means. 

35 8. Digital data conversion equipment as claimed in claim 7 characterized in that the resolution of a video 
signal of first standard system is lower than that of a video signal second standard system. 

9. Digital data conversion equipment as claimed in claim 7 characterized in that the resolution of a video 
signal of the first standard system is higher than that of a video signal second standard system. 
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10. Digital data conversion equipment, comprising: 

means for analyzing plural input data and performing clustering depending on a distribution state 
of said plural input data; 

means for generating class data associated with output data for each class on the basis of said 
plural input data and known output data; 

memory means for storing said class data at an address corresponding to said class; 

read-out means for reading out class data at an addres corresponding to class information sub- 
jected to clustering based on the plural input data composed of transmission picture elements; and 

output data generating means for generating thinned-out data based on output class data of said 
read-out means. 
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Fig. 2 
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Fig. 4 
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Fig* 6 
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